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Synopsis 


Our  research  is  focused  upon  new  methods  to  support  decision-making  on  acquisitions  in  a  system 
of  systems  (SoS).  We  have  improved  upon  prior  work  by  introducing  both  an  extension  to  the  previously 
developed  Computational  Exploratory  Model  (CEM)  and  a  decision-analysis  framework  that  addresses 
issues  of  identifying  optimal  collections  of  interconnected  systems.  An  Agent-Based  Model  (ABM) 
approach  extends  prior  CEM  capabilities  by  enabling  the  development  of  a  simulation  to  model 
interdependencies  between  constituent  systems  in  a  SoS.  A  robust  investment  portfolio  framework 
complements  the  approach  by  providing  a  means  to  objectively  balance  expected  performance,  cost,  and 
risks  in  identifying  optimal  collections  of  systems  that  constitute  a  SoS. 

Executive  Summary 

The  Department  of  Defense  (DoD)  has  recognized  the  need  for  effective  tools  in  managing 
collections  of  systems  that,  when  operating  in  a  networked  manner,  serve  to  provide  an  overarching  set  of 
capabilities.  Acquisition  management  of  these  systems  of  systems  (SoSs)  is  difficult  due  to  the  complex 
interdependencies  that  exist  between  constituent  systems,  both  across  the  programmatic  and  technical 
phases  of  development.  The  identification  of  developmental  risks,  their  consequences,  and  expected 
payoffs  (potential  gains  in  SoS-wide  capabilities)  are  equally  important  in  identifying  an  optimal  SoS 
architecture. 

Research  efforts  during  this  report  period  have  focused  on  extending  previous  frameworks  of  the 
Computational  Exploratory  Model  (CEM)  and  Markov-based  analytic  approach.  The  CEM  and  Markov 
approaches  were  developed  as  means  of  comparing  different  networks  of  systems  and  quantifying 
cascading  modes  of  risks  that  propagate  due  to  the  interconnectivities  that  exist  between  constituent 
systems.  The  cascading  risks  transcend  both  technical  and  programmatic  dimensions  of  acquisition 
management  and  often  produce  developmental  delays.  The  research  in  this  report  assumes  a  two-pronged 
approach.  The  first  introduces  an  Agent-Based  Model  (ABM)  that  extends  the  exploratory  nature  of  the 
prior  CEM  and  Markov  approaches  by  considering  the  dynamics  and  propagating  risks  beyond  their 
purely  probabilistic  setting.  ABMs  are  extremely  well  suited  to  simulating  complex  interactions  of  a  SoS, 
and  can  incoiporate  a  large  range  of  types  of  information  that  drive  the  interactive  behavior  between 
participating  “agents.” 

The  second  approach  complements  the  exploratory  nature  of  the  aforementioned  work  with  a 
decision-analysis  framework  that  is  centered  on  an  investment  portfolio  approach.  The  acquisition 
strategy  proposed  in  our  project  is  based  on  the  16  technical  management  and  technical  system¬ 
engineering  processes  outlined  in  the  Defense  Acquisition  Guidebook  (DAG;  U.S.  DoD,  2008a),  often 
referred  to  as  the  5000-series  guide,  and  the  DoD  Systems  Engineering  Guide  for  Systems  of  Systems 
(SoS-SE;  U.S.  DoD,  2008b)  guide.  Decisions  on  system  acquisitions  are  accomplished  by  adapting 
algorithmic  advances  in  robust  portfolio  optimization  techniques  from  financial  engineering  and 
operations  research,  to  deal  with  uncertainties  that  can  affect  optimal  selection  of  “portfolios”  of  systems 
in  a  SoS.  The  report  details  an  approach  to  SoS  acquisition  in  the  form  of  a  robust  optimization  problem 
that  balances  rewards  (potential  gain  in  SoS-wide  capabilities)  against  developmental  risks  and 
acquisition  cost.  The  approach  is  illustrated  by  using  a  simple  concept  problem  based  on  the  Littoral 
Combat  Ship  (LCS)  platform. 
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Outreach  and  Collaboration 

Work  documented  in  this  report  has  resulted  in  a  conference  publication  and  a  presentation  at  the 
NPS  Ninth  Annual  Acquisition  Research  Symposium  in  2012.  Resulting  interactions  have  produced  very 
valuable  feedback  on  the  merits  of  our  current  results  and  potential  further  development  of  the  portfolio 
approach.  The  symposium  especially  allowed  us  to  foster  closer  ties  and  exchanges  with  various  members 
of  the  NPS  community.  In  particular,  we  have  been  involved  in  collaborative  exchanges  with  Dr.  Tom 
Housel  and  Dr.  Jonathan  Mun,  who  are  actively  involved  in  the  development  of  portfolio-based 
approaches  for  acquisition  and  risk  management.  The  work  in  this  report  has  also  been  presented  at  the 
INFORMS  2012  Conference,  October  14-17,  in  Phoenix,  AZ.  The  conference  presentation  generated 
useful  feedback  from  attending  practitioners  in  the  operations  research  and  financial  engineering 
communities,  focused  on  ways  to  improve  and  further  develop  the  framework  presented  in  this  report. 

Introduction 

A  system  of  systems  (SoS)  consists  of  a  network  of  operationally  and  managerially  independent 
systems  that  work  synergistically  in  achieving  an  overarching  capability  (Maier,  1998).  This  confluence 
of  multiple  entities  gives  rise  to  an  emergent  behavior  that  may  not  be  explicitly  apparent  from  the 
individual  constituents.  The  SoS  paradigm  has  motivated  the  development  of  new  acquisition  strategies 
and  integration  of  individual  systems  to  better  address  the  issue  of  achieving  a  set  of  overall  capabilities 
instead  of  requirement-specific  metrics.  Acquisition  efforts  to  obtain  SoS  capabilities  for  projects  such  as 
the  U.S.  Army’s  Future  Combat  Systems  (FCS;  Gilmore,  2006)  and  the  U.S.  Coast  Guard’s  Deepwater 
System  have  been  met  with  a  large  degree  of  developmental  difficulties.  This  includes  a  combination  of 
vulnerabilities  in  the  developmental  stage  and  poor  management  oversight  that  often  leads  to  costly 
schedule  overruns  and,  ultimately,  cancellations.  These  large-scale  systems  are  often  developed 
incrementally  with  system-level  requirements  being  the  immediate  focus  of  attention.  Implicit 
consideration  is  given  to  the  overarching  objectives  of  the  intended  SoS  capability.  The  decoupled  and 
decentralized  nature  of  architecting  SoS  across  multiple  hierarchies  of  interdependencies  has  given  rise  to 
a  range  of  inefficiencies  and  warranted  the  pursuit  of  better  systems  engineering  practices  to  encompass 
SoS  principles. 

The  recognition  of  the  need  for  improved  methods  in  architecting  and  acquiring  systems  that 
comprise  a  SoS  has  led  to  further  research  in  developing  frameworks  that  maximize  SoS-wide  capabilities 
whilst  minimizing  cost  and  mitigating  risk.  Our  prior  efforts  under  NPS  Acquisition  research  program 
funding  have  introduced  the  concept  of  a  Computational  Exploratory  Model  (CEM) — a  discrete  event 
simulator  for  the  development  and  acquisition  process.  The  CEM  is  based  on  the  16  basic  technical 
management  and  technical  systems-engineering  processes  outlined  in  the  Defense  Acquisition  Guidebook 
(U.S.  DoD,  2008a) — referred  to  as  the  5000-series  guide.  The  method  also  considers  the  modified 
processes  in  accordance  with  the  Systems  Engineering  Guide  for  System-of-Systems  (SoS-SE;  U.S.  DoD, 
2008b)  that  adapts  the  5000-series  guide  processes  to  the  SoS  framework.  Research  work  in  the  present 
reporting  period  sought  to  provide  a  complementary  decision-making  tool  that  focuses  on  balancing 
capability  development  against  cost  and  interdependent  risks  through  the  use  of  Modem  Portfolio  Theory 
(MPT). 
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Extending  CEM  and  Markov:  Agent-Based  Modeling  Approach 

Research  conducted  in  the  previous  reporting  period  extended  the  CEM  work  to  include  a 
Markov  perspective  (Mane,  DeLaurentis,  &  Frazho,  2011)  that  utilizes  network-level  metrics  and 
computes  conditional  probabilities  that  relate  the  transmission  of  cascading  risks  between  nodes  (systems) 
in  an  interconnected  SoS  network.  The  Markov  extension  to  the  CEM  approach  allows  for  the  comparison 
of  alternate  SoS  architectures,  with  the  intention  of  mitigating  cascading  risks  and  identifying  SoS 
architectures  with  desired  overarching  capabilities.  Elowever,  both  the  CEM  and  Markov  approaches 
require  inputs  on  interaction  and  performance  in  the  general  SoS  setting.  Our  current  work  extends  the 
CEM  and  Markov  approaches  to  now  include  an  Agent-Based  Model  (ABM)  that  allows  for  the  modeling 
of  interacting  entities  as  discrete  “agents.”  Each  agent  has  a  predefined  set  of  behaviors  and  rules  of 
interaction.  The  modeling  and  simulation  of  these  interacting  agents  allows  for  a  more  generalized 
computational  framework  that  is  not  predominantly  based  on  fixed  probabilities,  but  rather,  on  a  much 
wider  range  of  possible  metrics  for  simulation.  The  fidelity  of  the  ABM  is  tuned  such  that  its  outputs  are 
well  suited  to  the  required  inputs  for  our  CEM,  Markov-based,  and  robust  portfolio  models.  We  motivate 
this  addition  with  the  simulation  of  a  naval  warfare  scenario  that  is  based  on  the  Littoral  Combat  Ship 
(LCS)  platform. 

Example  Application:  Naval  Warfare  SoS  Agent-Based  Model 

Full-scale  littoral  battle  scenarios  involve  a  Littoral  Combat  Ship  (LCS)  squadron  with  a  mix  of 
ships  and  accompanying  helicopters,  unmanned  aerial  and  surface  vehicles,  and  a  host  of  antagonist  units. 
Such  scenarios  exhibit  all  the  characteristics  of  a  SoS  problem  with  each  individual  asset  possessing 
operational  and  managerial  independence,  with  spatial  distribution,  and  evolving  with  time  to  exhibit 
emergent  behavior  as  a  result  of  system  interaction.  Agent-Based  Modeling  (ABM)  is  ideally  suited  to 
tackle  SoS  problems,  and  Purdue  University  has  its  very  own  MATLAB-based  simulation  framework 
called  Discrete  Agent  Framework  (DAF).  Elence  an  agent-based  simulation  model  was  developed  using 
DAF  to  capture  the  performance  of  the  Littoral  Combat  Ship  (LCS)  in  a  warfare  scenario  involving 
multiple  mission  threats.  The  ABM  itself  was  based  on  the  models  described  in  three  masters’  theses  from 
the  U.S.  Naval  Postgraduate  School  (Abbot,  2008;  Ozdemir,  2009;  Jacobson,  2010).  Military  and  industry 
experts  have  verified  that  the  agents,  that  constitute  the  SoS,  are  based  on  realistic  and  well-reviewed 
information. 

Enemy  Forces 

In  order  to  accurately  capture  how  LCS  will  perform  in  a  stressing  operational  environment,  a 
robust  scenario  containing  three  different  types  of  antagonist  units — missile  boats,  submarines,  and 
mines — was  developed.  Each  type  of  enemy  unit  was  assigned  a  home  position  at  the  start  of  the  scenario. 
The  missile  boats  and  submarines  exhibit  different  patrol,  detection,  and  engagement  strategies,  while  the 
mines  used  in  this  simulation  simply  detonate  whenever  an  agent  is  within  its  proximity  radius. 

Friendly  Forces 

The  LCS  was  designed  primarily  to  take  advantage  of  the  interchangeable  modularized  onboard 
packages.  Each  of  the  following  packages  was  modeled  in  the  simulation  scenario:  Surface  Warfare 
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(SUW),  Anti-Submarine  Warfare  (ASW),  and  Mine  Countermeasures  (MCM).  Apart  from  the  primary 
mission  package,  the  protagonist  forces  were  augmented  by  the  presence  of  a  host  of  supporting  entities 
which  included — MH-60R,  Unmanned  Aerial  Vehicles  (UAVs),  Unmanned  Surface  Vehicles  (USVs), 
and  Remote  Mine-Hunting  Vehicles  (RMVs).  Each  of  the  units  has  different  capabilities  and  tackles 
different  antagonist  units,  depending  on  the  LCS  mission  package  to  which  it  is  affiliated. 

Neutral  Forces 

Merchant  traffic  was  included  to  emulate  realism  in  the  scenarios  and  to  add  to  the  surface  clutter, 
thereby  making  detection  more  difficult  for  both  the  protagonist  and  antagonist  units. 

Goal 


The  mission  of  the  LCS  fleet  was  to  clear  the  waters  of  all  threats,  while  incurring  the 
minimum  number  of  friendly  casualties.  The  factors  that  played  an  important  role  in  this  simulation  are 
the  number  of  enemy  platforms,  the  number  and  type  of  LCS,  the  detection  probability  for  the  friendly 
sensors,  and  the  kill  probability  of  friendly  weapons. 

Results 


Multiple  simulation  runs  were  carried  out  in  order  to  analyze  the  battle  scenario  and  the  agent 
behavior.  Figure  1  displays  the  starting  and  the  emergent  scenario  that  arises  for  a  typical  run  case. 


The  Naval  Warfare  Simulation,  depicted  in  Figure  1,  provides  a  good  insight  into  the  workings  of 
agent-based  modeling  and  serves  as  an  ideal  exploratory  model  to  glean  the  mission  performance  for  a 
naval  warfare  scenario.  The  model  captures  salient  emergent  behavior  and  properties  that  result  from  the 
collective  interaction  among  the  system  components  and  provides  inroads  in  capturing  the  performance 
and  the  shortcomings  of  the  LCS. 

Multiple  simulation  runs  were  carried  out  in  order  to  analyze  the  battle  scenario  and  the  agent 
behavior.  Table  1  depicts  the  results  from  a  typical  simulation  run. 
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Table  1.  Results  From  a  Typical  Simulation  Run 


Agent 

Terminated 

Agent 

Location 

Time 

Step 

Terminator 

Terminator 

Location 

Weapon 

Used 

Mine  5 

(75,40) 

38 

MIW 

MH60 

(78.85,32.18) 

Clearance 

Mine  4 

(65,50) 

200 

MIW 

MH60 

(68.7,  42.9) 

Clearance 

Mine  3 

(55,60) 

272 

MIW 

MH60 

(58.70,49.89) 

Clearance 

Mine  2 

(35,65) 

621 

MIW 

MH60 

(40.58,65.77) 

Clearance 

MBS  1 

(17.8,111.18) 

623 

SUW  LCS 

(30.88,96.62) 

NLOS 

MBS  2 

(17.96,110.09) 

635 

SUW  LCS 

(28.25,99.63) 

NLOS 

MBS  3 

(18.65,112.08) 

642 

SUW  LCS 

(26.72,101.39) 

NLOS 

MBS  4 

(19.2,112.56) 

651 

SUW  LCS 

(24.75,103.66) 

NLOS 

MBS  5 

(18.10,108.88) 

659 

SUW  LCS 

(22.98,105.66) 

NLOS 

Mine  1 

(15,75) 

665 

MIW 

MH60 

(20.32,73.5) 

Clearance 

ASW  LCS 

(44.72,79.73) 

763 

Submarine 

(44.69,  79.69) 

Torpedo 

The  simulation  results  reveal  the  expected  outcome  for  missile  boats  and  mines  based  on 
intuition,  and  an  unanticipated  outcome  for  the  ASW  LCS  due  to  emergent  properties.  The  SUW  MH- 
60R  helicopter  detects  missile  boats  early  in  the  scenario  due  to  its  high  velocity  and  detection  radius.  The 
SUW  MH-60R  maintains  a  standoff  distance  of  20  nautical  miles  (nm)  from  the  detected  missile  boats 
and  passes  on  the  position  information  to  the  SUW  LCS.  Guided  by  the  antagonist’s  position  information, 
the  SUW  LCS  was  able  to  terminate  each  of  the  missile  boats  using  its  Non-Line-of- Sight  Launch  System 
from  distances  ranging  from  6-20  nm  without  suffering  any  fire  in  return.  The  missile  boats  could  not  get 
close  enough  to  the  LCS  to  use  their  weapons,  which  had  a  very  short  range  of  1  nm.  The  MIW  LCS 
detects  the  mines  and  passes  on  the  position  information  to  the  MIW  MH-60  and  the  other  LCSs,  so  that 
they  can  avoid  colliding  with  the  mines.  The  MIW  LCS  maintains  a  standoff  distance  of  3  nm  from  the 
detected  mines,  while  the  MIW  MH-60,  guided  by  the  antagonist’s  position  information,  terminates  each 
of  the  mines  using  its  clearance  missiles. 

The  termination  of  the  ASW  LCS  at  the  hands  of  the  submarine  was  an  emergent  and  non- 
intuitive  result.  The  ASW  LCS  does  not  have  the  means  to  terminate  submarines  and  is  restricted  to 
maintain  a  standoff  distance  of  10  nm  from  a  detected  submarine  and  to  pass  on  the  submarine’s  location 
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to  the  accompanying  MH-60R.  Once  the  detected  submarine  is  within  the  weapon  range  of  the  ASW  MH- 
60R,  it  fires  torpedo  missiles  on  the  submarine;  however,  because  of  the  limited  number  of  torpedo 
missiles  onboard  the  ASW  MH-60R,  and  the  probability  of  termination  associated  with  the  toipedo 
missiles,  the  ASW  MH-60R  either  missed  its  target  or  could  only  injure  the  submarine,  before  it  ran  out 
of  ammunition.  The  submarine  was  then  able  to  terminate  the  ASW  LCS  before  the  ASW  MH-60R  could 
reload  its  torpedo  missiles.  Once  the  ASW  LCS  was  terminated,  the  ASW  MH-60R  could  no  longer 
reload  and  the  submarine  was  able  to  escape. 

The  Naval  Warfare  LCS  example  illustrates  the  salient  features  and  versatility  of  modeling 
complex  SoS  architectures  using  an  agent-based  approach.  The  ABM  approach  facilitates  analysis  of 
complex  dynamics  of  interacting  agents  while  providing  the  overarching  capabilities  that  emerge  from  the 
interaction  of  complex  agents.  The  DAF  implementation  used  in  this  research  allows  for  easy 
instantiation  of  scenarios,  be  they  operational  (such  as  in  this  LCS  example),  or  even  developmental.  The 
simulation  of  developmental  scenarios  would  be  representative  of  the  CEM  and  Markov  frameworks  that 
have  focused  on  programmatic  disruptions.  The  agent  model  simulated  in  our  LCS  scenario,  while 
operational  in  nature,  can  easily  be  extended  to  include  such  simulation  as  well. 

SoS  Acquisitions:  Robust  Investment  Portfolio  Approach 

Portfolio  management  techniques  have  been  successfully  used  to  address  strategic-level  asset 
acquisition  and  are  extendable  to  include  multi-period  considerations.  Real  options  analysis,  for  example, 
has  shown  effectiveness  across  various  industries  to  evaluate  discrete,  long-term  investment  strategies. 
The  work  by  Komoroski,  Housel,  Horn  &  Mun  (2006)  has  developed  a  methodology  that  addressees 
strategic  financial  decisions  through  an  eight-phase  process  using  a  toolbox  of  financial  techniques — 
including  portfolio  optimization  techniques.  Such  frameworks  are  geared  towards  financial  uncertainty 
considerations  of  strategic  projects  and  do  not  explicitly  address  technical  architecture  and/or  evolving 
SoS-wide  capabilities. 

Figure  2  is  a  simplified  adaptation  of  the  “Wave  model”  structure  that  has  been  advocated  as  a 
conceptual  model  for  guiding  SoSE  in  the  literature  (Dahmann,  Rebovich,  Lowry,  Lane,  &  Baldwin, 
2011).  The  purpose  of  the  Wave  model  is  to  identify  SoS  artifacts  and  their  employment  in  the 
engineering  of  SoS  architectures  (and,  ultimately,  in  acquisition  of  component  systems).  Our  research 
employs  a  lexicon,  denoted  by  Greek  letters,  that  indicates  hierarchical  levels  in  a  system  architecture, 
with  the  alpha  (a-level)  being  the  lowest  level  of  operation,  and  moving  up  towards  higher  levels  of  the 
hierarchy  from  there — the  lexicon  is  adapted  from  literature  on  a  SoS  taxonomy  guide  (DeLaurentis, 
Crossley,  &  Mane,  2011). 

The  Wave  model  is  adapted  here  to  include  hierarchy  and  time  scale  that  ranges  from  the  broad, 
overarching  objectives  that  are  strategic  in  nature  (y-level)  to  the  tactical  aspects  of  individual  system 
acquisition  (a-level).  Research  in  this  report  addresses  the  p-level  SoS  portfolio  development  stage  that 
evaluates  candidate  systems  and,  consequently,  selects  a  portfolio  of  interdependent  systems  to  fulfill 
overarching  SoS  capability  objectives.  The  idea  is  to  still  maintain  compliance  with  the  “top-down 
integration,  bottoms-up  implementation”  paradigm  that  is  part  of  the  Wave  model  implementation. 
Finding  optimal  portfolios  assists  in  SoS  architecting  (e.g.,  ensuring  capabilities  are  met),  while  also 
improving  acquisition  decisions  for  key  component  systems. 
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Figure  2.  The  Context  of  the  “Wave  Model”  (Dahmann,  2011)  in  Acquisition 

Decision-Making 

The  framework  proposed  in  this  report  does  not  attempt  to  replace,  but  rather  to  complement, 
existing  methodologies  by  more  directly  addressing  issues  of  integration  and  acquisition  from  a  robust 
portfolio  theory  standpoint.  Robust  portfolio  management  methodologies  have  been  widely  used  by 
financial  engineering  practitioners  to  manage  portfolios  in  the  face  of  market  volatility  and  uncertainties. 
In  the  present  context,  such  quantitative  guidance  is  important  for  providing  acquisition  groups  with  the 
means  of  performing  acquisition,  integration,  and  development  decision-making  in  the  mist  of  evolving 
capability  requirements. 

Our  approach  seeks  to  leverage  tools  from  robust  portfolio  methods  to  improve  acquisition, 
integration,  and  development  decisions  while  retaining  advantages  in  balancing  systems  acquisition 
against  evolving  capability  requirements.  The  research  complements  the  notion  of  open  architectures  and 
modularity  by  providing  a  means  of  managing  the  possible  combinations  of  connectible  assets,  and 
indirectly  promoting  competitive  development  of  these  interchangeable  assets. 

Open  Architectures,  Competition,  and  Modularity 

Open  architecture  (OA)  involves  the  design  and  implementation  of  systems  that  conform  to  a 
common  and  unified  set  of  technical  interfaces  and  business  standards.  This  form  of  architecture  results  in 
the  development  of  modular  systems  and  increases  opportunities  for  innovation  and  rapid  development  of 
new  technologies  that  can  be  readily  integrated/swapped  into  current  architectures.  The  Littoral  Combat 
Ship  (LCS)  program,  for  example,  has  recognized  the  need  for  multi-vendor  acquisitions  and  OA 
implementations  to  ensure  greater  technological  adaptability.  The  LCS  program  exploits  the  benefits  of 
dual  award  contracting  under  fixed-price  initiatives  (FPI),  along  with  rapid  technology  insertion  processes 
and  open  architectures,  to  fulfil  evolving  technological  and  mission  requirements  of  littoral  warfare.  The 
combination  of  dual  contracting  and  system  modularity  helps  achieve  the  necessary  cost  reductions  while 
maintaining  a  greater  degree  of  adaptability  towards  changing  mission  requirements  (“LCS,”  2011;  U.S. 
GAO,  2007b;  Gansler  &  Lucyshyn,  2008).  Although  the  platform  is  not,  strictly  speaking,  a  SoS,  it 
nevertheless  is  a  representative  microcosm  of  what  constitutes  a  SoS.  The  LCS  platform  carries  many 
comparable  salient  features  such  as  the  confluence  of  multiple  (sub)  systems  that  work  cohesively  to 
achieve  required  capabilities. 
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Figure  3.  Littoral  Combat  Ship  Layout 


(“LCS,”  2011) 

The  benefits  of  open  architectures  and  competitive  contracting  are  intuitively  clear  and  have  been 
shown  to  generate  notable  cost  savings  as  exhibited  in  previous  development  projects  such  as  Joint  Direct 
Attack  Munitions  (JDAM;  U.S.  GAO,  2007b).  However,  system  integrators  and  program  managers  are 
often  faced  with  the  challenge  of  leveraging  the  potential  benefits  of  introducing  new  and  improved 
systems  against  potential  risks  associated  with  developmental  disruptions  and  cost  considerations. 
Although  the  LCS  program  had  significant  success  through  the  dual  contracting  scheme,  it  still 
experienced  cost  overruns  due  to  a  variety  of  problems.  The  problems  included  risks  from  a  simultaneous 
design  and  build  strategy  due  to  schedule  constraints,  unrealistic  budget  expectations,  and  market  risk 
from  the  greatly  increased  price  of  steel  during  the  development  period  (O’Rourke,  2011).  There  have 
also  been  revisions  in  the  requirements  of  fleet  capabilities  and  refocusing  of  intended  capabilities 
(O’Rourke,  2011).  The  robust  investment  portfolio  approach,  we  believe,  as  formulated  in  this  research 
work,  provides  a  beneficial  framework  that  takes  full  advantage  of  the  “open  architecture”  approach  to 
system  and  capability  development.  It  offers  the  combinatorial  nature  of  “mixing  and  matching”  feasible 
collections  of  systems  while  balancing  risk  (developmental),  rewards  (potential  capabilities),  and  cost  of 
the  overall  SoS. 

Concept  Acquisition  Portfolio:  Littoral  Combat  Ship  Example 

Littoral  Combat  Ships  are  designed  and  developed  by  two  primary  contractors — General  Dynamics 
and  Lockheed  Martin — as  a  result  of  the  Navy’s  dual  contract  award  strategy  that  seeks  to  minimize  costs 
through  competitive  contracting.  The  ships  are  designed  to  serve  as  primary  units  in  close  coastal  littoral 
warfare  and  take  advantage  of  modularized  onboard  packages  (systems)  that  are  interchangeable  for 
different  operational  requirements.  These  packages  include  the  Anti-Submarine  Warfare  (ASW),  Mine 
Countermeasure  (MCM)  and  Surface  Warfare  (SUW)  packages.  More  recent  developments  have  seen  the 
introduction  of  an  irregular  warfare  package  for  assistance  and  general  support  missions.  We  use  this  LCS 
example  to  demonstrate  our  work  since  it  is  a  case  where  the  objective  is  to  achieve  desired  combat 
effectiveness  and  operational  capabilities  while  minimizing  cost  and  development  risk.  The  simple  model 
inputs  and  characteristics  are  described  in  Table  1. 
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Table  2.  Individual  System  Information  for  LCS  Example 


System  Capabilities 

System  Req 

Dev. 

Time 

Acq. 

Cost 

Weapon 

Threat 

Anti  Mine 

Comm. 

Air/Sea  State 

Air/Sea 

Comm. 

(Years) 

<$) 

Strike 

Detection 

Detection 

Capacity 

Capacity 

State 

Range 

Range 

Speed 

Package 

ASW 

Variable  Depth 

0 

50 

0 

0 

0 

0 

250 

125 

3 

3000000 

Multi  Fen  Tow 

0 

40 

0 

0 

0 

0 

150 

75 

2 

2000000 

Lightweight  tow 

0 

30 

0 

0 

0 

0 

100 

50 

4 

4000000 

MCN 

RAMCS  II 

0 

0 

40 

0 

0 

3 

200 

100 

1 

1000000 

ALMDS  (MH-60) 

0 

0 

30 

0 

0 

4 

100 

50 

2 

2000000 

SUW 

N-LOS  Missiles 

25 

0 

0 

0 

0 

0 

200 

100 

3 

3000000 

Griffin  Missiles 

3 

0 

0 

0 

0 

0 

100 

50 

4 

4000000 

Sea  fra  me 

Package  System  1 

0 

0 

0 

400 

4 

0 

0 

0 

3 

3000000 

&  Combat 

Package  System  2 

0 

0 

0 

300 

4 

0 

0 

0 

4 

4000000 

Manage, 

Package  System  3 

0 

0 

0 

250 

3 

0 

0 

0 

5 

5000000 

Table  2  is  a  hypothetical  and  simplified  catalogue  of  individual  systems  available  to  the  Navy  in  its 
pursuit  of  achieving  desired  capabilities.  Although  the  numbers  are  hypothetical  and  do  not  explicitly 
illustrate  real  data,  the  salient  features  of  considering  capabilities,  requirements,  and  risk  in  acquisition 
problems  are  still  preserved.  Table  2  lists  systems  that  are  available  for  each  of  the  three  mission 
packages — ASW,  MCM,  SUW — along  with  an  individual  rating  of  system  capabilities  and  requirements 
for  the  systems  to  operate.  Additionally,  Table  2  provides  the  system  development  time  and  associated 
acquisition  costs.  Systems  that  are  unable  to  provide  a  particular  capability  (or  do  not  have  a  particular 
requirement)  have  a  zero  entry.  Although  the  sea  frame  is  typically  a  single  system,  the  current  sample 
problem  couples  the  sea  frame  with  battle  management  software  as  a  base  system  that  provides  intra 
system  capabilities.  The  development  of  these  systems  is  based  on  a  projected  time  schedule  that  is 
inherently  subject  to  overruns  and  risk.  This  element  is  captured  in  the  covariance  matrix  shown  in  Table 
3. 
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Table  3.  System  Interdependency  and  Development  Risk  (Covariance) 


Variable  Depth 

Multi  Fen  Tow 

Lightweight  tow 

RAMCS  II 

ALMDS  (MH-60) 

N-LOS  Missiles 

Griffin  Missiles 

Package  System  1 

Package  System  2 

Package  System  3 

Variable  Depth 

0.1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Multi  Fen  Tow 

0 

1  0*6 

0 

0 

0 

0 

0 

0.1 

0.1 

0 

Lightweight  tow 

0 

0 

0.2  | 

0 

0 

0 

0 

0.3 

0 

0.2 

RAMCS  II 

0 

0 

0 

0.3 

0.1 

0 

0 

0 

0.2 

0 

ALMDS  (MH-60) 

0 

0 

0 

0.1 

0.1 

0 

0 

0 

0 

0.3 

N-LOS  Missiles 

0 

0 

0 

0 

0 

0.5 

0.2 

0 

0.1 

0 

Griffin  Missiles 

0 

0 

0 

0 

0 

0.2 

0.3 

0 

0 

0 

Package  System  1 

0 

0.1 

0.3 

0 

0 

0 

0 

0.5 

0 

0 

Package  System  2 

0 

0.1 

0 

0.2 

0 

0.1 

0 

0 

O 

L° 

Package  System  3 

0 

0 

0.2 

0 

0.3 

0 

0 

0 

0 

0.2 

Table  3  shows  the  risk  and  interdependency  aspects  of  the  decision  process.  The  diagonal  terms 
represent  the  variance  (degree  of  deviation  from  expected  time)  in  development  time.  The  off-diagonal 
terms  are  the  variances  due  to  interdependencies  between  individual  systems  that  have  commonly 
developed  subsystems.  For  example,  since  the  N-LOS  and  Griffin  missile  systems  are  both  developed  by 
Northrop  Grumman,  it  is  conceivable  that  they  have  common  parts  or  undergo  similar  processes  in 
development  and  manufacturing.  The  covariance  value  of  0.2  in  the  non-diagonal,  therefore,  represents 
joint  development  risk  in  terms  of  development  time  variance  (e.g.,  0.2  years),  due  to  interdependencies 
between  the  two  systems. 

Estimation  of  the  covariance  matrix  entries  can  come  directly  from  manufacturing  and 
development  data.  In  the  case  of  new  systems,  the  quantities  can  be  estimated  heuristically  using  basic 
rules  similar  to  those  used  in  project  management  techniques  such  as  PERT  and  other  CPM  methods 
(Blanchard  &  Fabrycky,  2005).  The  CEM  and  Markov  analytical  frameworks  from  prior  research  efforts 
present  a  means  of  generating  surrogates  of  these  covariance  matrices,  through  identification  of 
interdependencies  (covariance)  between  systems. 

The  entries  of  the  matrix  in  Table  2  are  typically  inferred  from  data;  in  this  case,  the  values  are 
hypothetically  developed  for  the  concept  example  problem.  Most  of  the  individual  systems  do  not  bear 
many  interdependencies,  with  the  exception  of  the  sea  frame  and  combat  management  support  systems 
that  are  interlinked  more  explicitly  to  other  listed  systems  in  Table  2. 
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Investment  Model  Formulation  and  Solution 


The  problem  statement  for  the  given  acquisition  problem  is  formulated  as  a  mathematical 
optimization  problem,  which  requires  the  definition  of  the  objective  function  and  constraints.  The 
objective  function  is  the  equation  that  describes  the  primary  metric  to  be  optimized.  This  typically 
translates  to,  for  example,  the  maximization  of  profits  or  minimization  of  costs/risk  in  the  commercial 
sense.  The  second  important  aspect  deals  with  the  formulation  of  constraints,  which  are  equations  that 
typically  describe  resource  (e.g.,  time,  cost)  constraints  on  the  system  and  can  be  manipulated  to  reflect 
the  salient  conditions  of  the  problem  to  be  solved.  The  investment  portfolio  problem  presented  in  the 
formulation  shown  in  Equations  1-9  (also  known  as  the  Markowitz  formulation)  seeks  to  maximize  the 
aggregate  capabilities  of  a  SoS  architecture  while  minimizing  the  cumulative  effect  of  cost, 
developmental  time,  and  integration  risks.  The  mathematical  model  for  the  concept  problem  can  be 
written  as  the  following: 


max 


I 

V  ? 


sqc-Rc 

R 


■wXL 


-HK) 


u  q 


2KA) 


(1) 


XB  C 

X 'l  =  — - — —  (Portfolio  Fractions) 

Budget 

^  Cq  X'l  +  s  =  Budget  (Budget  Constraint) 

9 

'y'SaCXB  >  ^ SqR  Xq  (Satisfy  All  System  Requirements) 

9  9 

Xf  +  Xf  +  Xf  =  1  (ASW  System  Compatibility) 

X l  +  XB  =  1  (MCM  System  Compatibility) 

XB  +  XB  =  1  (SUW  System  Compatibility) 

XB  +  XB  +  Xf0  =  1  (Package  System  Compatibility) 

XB  e  {0,1}  (binary) 


(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

(8) 
(9) 


where:  S„ 


K 

w 

K 

c 


:  system  (^)’s  value  for  capability  (c) 

:  system  (q)’s  value  for  requirement  (r) 

:  reference  value  for  baseline  desired  SoS  capability  (c) 

:  weighting  factor  vector  that  weights  importance  across  desired  SoS  capabilities 
:  decision  variable  of  selecting  system  (q) 

:  decision  variable  of  budget  fraction  that  selecting  system  ( q )  represents 
:  monetary  cost  of  acquiring  system  (q) 
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E..  :  covariance  matrix  of  risk  (variances  of  development  and  integration  time  for 

systems) 

A,  :  risk  aversion  factor 

£  :  budget  remainder 

Budget  :  total  acquisition  budget 

The  mathematical  model  shown  by  Equations  1-9  represents  the  formulation  of  a  traditional  single- 
stage  optimization  problem  that  is  typical  of  operations  research  and  financial  engineering  circles.  The 
current  form  for  the  portfolio  model  at  hand  is  known  as  a  quadratic  integer  program  (QIP)  and  is  based 
on  the  Markowitz  formulation  that  seeks  to  generate  optimal  portfolios  that  balance  potential  expected 
rewards  against  risk.  Equation  1  is  the  objective  function.  The  objective  is  to  maximize  overall  capability 
while  minimizing  cost  and  development  risk.  Equation  2  is  the  fraction  of  the  budget  invested  in 
individual  systems.  Equation  3  is  the  budgeting  constraints,  where  the  sum  of  all  investments  in 
individual  systems  (and  savings)  must  be  equal  to  the  total  budget  allotted.  Equation  4  ensures  that  all 
requirements  of  individual  systems  must  be  met.  Equations  5-7  are  the  individual  system  compatibilities. 
In  Equations  5-7,  this  translates  to  the  selection  of  one  system  from  each  mission  package  (ASW,  MCM, 
SUW)  and  a  sea  frame  and  combat  management  package  that  services  the  mission  modules.  These 
packages  are  mutually  exclusive  and,  therefore,  warrant  a  total  selection  of  summation  equal  to  1,  which 
ensures  that  no  two  packages  per  category  are  selected  to  satisfy  the  respective  requirements.  The 
covariance  matrix,  as  denoted  by  ,  represents  variations  in  development  time  due  to  system 

interdependencies. 

The  mathematical  programming  formulation  of  Equations  1-9  requires  the  covariance  matrix  to  be 
positive  definite  (positive  eigenvalues);  in  the  context  of  financial  applications,  this  means  that  there 
should  not  be  a  combination  of  assets  that  result  in  a  zero  net  risk  condition.  Given  that  the  entries  of  the 
covariance  matrix  are  estimated  individually,  there  is  a  possibility  that  the  resulting  matrix  can  violate  the 
positive  definite  constraint;  however,  this  can  be  corrected  using  a  procedure  that  attempts  to  “find”  the 
closest  positive  definite  covariance  matrix  that  can  represent  the  existing  covariance  matrix.  The 
procedure  requires  solution  of  the  following  optimization  problem: 


min  dF  |X,X) 

(10) 

(11) 

Xec; 

(12) 

Equation  10  is  the  objective  function  that  seeks  to  minimize  the  difference  between  the  provided 

covariance  matrix,  X  ,  and  the  closest  positive  definite  matrix,  X  (decision  variable).  Equation  1 1 
enforces  the  condition  that  the  entries  of  the  matrix  are  bounded  between  some  pre-determined  upper  and 
lower  bounds.  Equation  12  ensures  that  the  covariance  matrix  exists  in  the  cone  of  positive  semi-definite 
matrices,  C”.  The  formulation  presented  in  Equations  10-12  can  be  further  altered  through  introduction 

of  relevant  constraints,  to  further  tailor  the  characteristics  of  the  covariance  matrix.  For  example, 
constraints  that  enforce  non-connections  between  systems  that  are  known  to  not  have  any  direct 
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connectivity  can  be  enforced  to  ensure  that  the  closest  positive  definite  covariance  matrix  also  adheres  to 
such  a  condition. 

Investment  Portfolio  Robustification 

It  is  well  known  in  financial  engineering  circles  that  the  Markowitz  formulation,  as  used  in  the 
simplified  LCS  scenario,  is  sensitive  to  changes  in  estimated  quantities  of  the  covariance  matrix  (system 
interdependencies)  and  expected  return  (system  performance).  The  sensitivity  due  to  poor  covariance 
estimations  can  result  in  highly  inefficient  portfolios  due  to  errors  in  estimation  or  market  shifts.  Such 
sensitivity  issues  have  prompted  the  development  of  a  variety  of  robust  methods  in  portfolio  analysis  to 
ensure  that  the  chosen  portfolio  of  assets  is  stable  against  potential  changes  in  market  conditions/expected 
volatility. 

The  current  portfolio  formulation  in  Equations  1-9  can  be  reformulated  using  robust  optimization 
techniques;  this  includes  Semi-Definite  Programming  (SDP)  approaches  (Fabozzi,  Kolm,  Pachamanova, 
&  Focardi,  2007;  Tutuncu  &  Comuejols,  2007)  that  are  extensions  of  modem  portfolio  and  control 
theory.  The  reformulation  allows  for  possible  changes  in  estimated  quantities  (e.g.,  due  to  market  shifts  in 
pricing,  volatility,  system  interdependencies)  to  be  accounted  more  explicitly  as  uncertainty  sets.  The 
resulting  portfolio  allocation  will  not  change  appreciably  even  if  salient  estimated  quantities  or  benefits 
change  (within  prescribed  limits).  In  the  context  of  an  acquisition  problem,  the  use  of  a  robust  formulation 
translates  to  reduced  costs  associated  with  capability  estimation  errors,  development  time  volatility,  and 
changing  requirement  conditions. 

The  general  form  of  the  portfolio  problem  in  this  research  can  be  reposed  as  a  robust  optimization 
problem,  given  by  the  following  form  (Fabozzi  et  al.,  2007;  Tutuncu  &  Comuejols,  2007): 


max  jmin { //,x}  -  A,  max  |xrEx|| 

(13) 

A X>B 

(14) 

CX=D 

(15) 

A  >  0,  A  >  0 

(16) 

Us{p)  =  {m  \  Mi  ~Mi  \<Sni  = 

(17) 

Tl  <  E  <  T.u 

(18) 

Although  the  complexity  of  the  optimization  problem  increases,  it  is  nevertheless  very  amenable 
to  a  collection  of  numerical  methods  that  provide  good  computational  performance  for  realistic  portfolio 
problems,  especially  portfolios  with  high  volatility  (Fabozzi  et  al.,  2007).  Equation  13  denotes  the  robust 
form  of  the  objective  function  in  (Equation  1).  Equations  14  and  15  are  the  generalized  linear  form  that 
represents  the  linear  relationships  in  Equations  6-11.  Equations  17  and  18  are  the  uncertainty  bounds  of 
the  performance  (capability)  and  operational  risk  due  to  interdependencies  in  each  system.  The  portfolio 
formulation  of  the  LCS  sample  problem  as  shown  in  Equations  1-9  is  rewritten  to  now  incoiporate  the 
uncertainties  that  are  associated  with  estimation  of  the  covariance  matrix  and  the  effective  capabilities  of 
individual  systems.  The  demonstration  LCS  problem  assumes  an  uncertain  covariance  matrix  and  utilizes 
the  conversion  methodology  as  detailed  in  literature  (Fabozzi  et  al.,  2007)  to  convert  the  problem  into  an 
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SDP.  Equations  1-9  are  adapted  into  the  robust  framework  in  Equations  13-18  to  yield  the  following 
SDP: 


max 


R 


V  »  V 


R 


w-Xi 


■A 


AZ)-(A£>}-X  (CX) 


XB  C 

XFn  = — - — —  (Portfolio  Fractions) 

’  Budget 

YCqXBq  +  s  =  Budget  (Budget  Constraint) 

9 

^SqCX:  >  ^ SqR XRq  (Satisfy  All  System  Requirements) 

9  9 

Xf  +  Xf  +  =  1  (ASW  System  Compatibility) 

+  X s  =  1  (MCM  System  Compatibility) 

Xf*  +  X7S  =  1  (SUW  System  Compatibility) 

Xl  +  XB  +  Xf0  =  1  (Package  System  Compatibility) 


A-A  x; 
XFq'  1 


±  0  (Linear  Matrix  Inequality) 


X*  g  {0,1}  (binary) 

XFq  e  □  " 

where 


(19) 


(20) 

(21) 

(22) 

(23) 

(24) 

(25) 

(26) 

(27) 

(28) 
(29) 


A,  A ,E,Z  :  dual  variables  associated  with  covariance  term 
(  ■XL  'j  :  denotes  trace  of  the  product  of  A  and  X 


Equations  19-28  retain  most  off  the  original  form  from  Equations  1-9.  The  exceptions  are  that  the 
uncertainty  in  the  covariance  matrix  and  absolute  value  bounds  on  the  capability  weighting  vector,  w,  are 
reintroduced  via  exploitation  of  the  dual  form  of  the  problem,  now  an  SDP.  The  reparametization  of  the 
problem  as  an  SDP  manifests  as  additional  terms  in  the  objective  function  and  constraints,  namely, 
Equations  27  and  28.  Equation  27  is  a  linear  matrix  inequality  and  enforces  the  condition  of  positive 
definiteness  due  to  the  symmetry  of  the  matrix  and  positive  values  of  all  variables  in  the  problem. 

The  uncertainty  set  of  the  covariance,  as  defined  in  Equation  15,  is  assumed  to  be  +/-  10%  of 
each  respective  entry  in  the  matrix.  This  is  arbitrarily  chosen  for  this  LCS  demonstration  problem; 
however,  real-world  problems  will  require  the  estimation  of  these  bounds  from  statistical  measures  such 
as  through  the  use  of,  say,  confidence  intervals.  Additional  measures  such  as  factor  models  can  be  used 
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to  estimate  the  values  of  the  covariance  with  respect  to  relevant  drivers;  for  a  SoS  problem,  metrics  such 
as  the  TRL  and  SRL,  which  directly  relate  to  project  risk,  may  be  used. 

Littoral  Combat  Portfolio:  Results 

The  primary  results  are  shown  by  varying  the  risk  aversion  parameter,  A  ,  each  time  to  generate  the 
robust  performance  efficiency  frontier.  Increasing  this  parameter  increases  the  portfolio’s  aversion  to  risk. 
The  increase  in  A  forces  the  portfolio  to  select  systems  that  have  lower  risk  and,  consequently,  results  in 
a  lower  performance  of  the  SoS  (Figure  4). 
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Figure  4.  Robust  Portfolio  Efficiency  Frontier 
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Portfolio  Capabilities 


Portfolio  1  Portfolio  2  Portfolio  3 


■  ASW 

■  MCN 

■  SUW 


Figure  5.  System  Capabilities 

In  Figure  4,  each  point  on  the  frontier  is  a  portfolio  that  corresponds  to  a  chosen  level  of  risk 
aversion  and  shows  the  amount  of  variance  associated  with  it.  The  higher  the  risk  aversion,  the  lower  the 
expected  SoS  performance  due  to  the  trade-off  in  choosing  say,  older,  more  reliable  technology  over 
newer  technologies  with  lower  TRL  values.  The  table  within  Figure  4  shows  the  portfolio  allocation  for 
each  of  the  three  critical  points  on  the  frontier,  with  the  corresponding  breakdown  in  capabilities  for  each 
portfolio  of  systems  shown  in  Figure  5.  Typical  efficiency  frontiers  will  have  more  points  due  to  the 
combinatorial  possibilities  of  systems  available  to  the  portfolio  selection  process.  One  system  is  common 
across  all  three  portfolios,  which  indicates  that  this  system  has  high  performance  and  relatively  low  risk. 
Some  systems,  however,  exhibit  increased  SoS-wide  performance  but  with  added  intersystem  risk.  The 
trade-off  between  individual  SoS  capabilities  (ASW,  MCM,  SUW)  and  risk  is  shown  in  Figure  4.  The 
analysis  as  shown  in  Figures  4  and  5  is  useful  for  acquisition  practitioners  to  determine  the  appropriate 
balance  of  SoS-wide  performance  against  developmental  risk. 

Multiple  Measures  of  Risk 

The  portfolio  analysis  method  in  this  work  can  be  extended  to  include  multiple  measures  of  risk, 
where  multiple  covariance  matrices  are  used  to  represent  the  connectivity  between  systems,  based  on 
different  layers  of  risk  in  developing  capabilities.  For  example,  in  the  case  of  considering  risks  in  the 
weapons  and  communications  layers  separately,  the  formulation  can  be  adapted  to  include  the  following 
constraints: 


X; 


y  Y 

^  weaspons 


F 


<  <T 


weapons 


(30) 
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•Jxn- 


communications 


<  <J 


communications 


(31) 


Equations  30  and  31  represent  the  risk  as  denoted  in  Equation  1,  but  now  for  two  separate  forms 
of  risk.  Each  equation  is  subject  to  a  limit  of  variance,  crweapons / &commimications  that  is  determined  by  the 

acquisition  practitioner.  Figure  6  shows  (using  arbitrary  covariance  matrices  to  represent  risks  for  each 
layer)  the  topology  of  performance  against  each  measure  of  risk  (weapons/communications)  for  the  LCS 
case.  An  increase  in  risk  for  each  dimension  of  capability  presents  a  potential  increase  in  the  SoS-wide 
capability.  The  discrete  jumps  in  performance  are  reflective  of  the  discrete  nature  of  including  individual 
systems  in  the  portfolio  of  systems  that  constitutes  the  SoS  architecture. 


Variance  Risk  Measure  (Comm)  Variance  Risk  Measure  (Weapons) 


Figure  6.  Example  Portfolio  Assessment  With  Two  Risks 

Future  Work 

Formulating  a  Dynamic  Investment  Policy 

The  multi-period  portfolio  problem  draws  upon  a  rich  history  of  algorithmic  development,  as  noted  in 
literature  both  in  the  operations  research  and  financial  engineering  circles.  Sequential  decision-making  is 
known  more  broadly  as  dynamic  programming,  and  adapts  control  theory  methodologies  to  dynamic 
management  of  resources  in  the  interest  of  maximizing  (or  minimizing)  some  given  metric.  The 
construction  of  an  appropriate  dynamic  policy,  in  the  context  of  an  acquisition  management  problem, 
translates  to  balancing  decisions  on  acquisitions  and  their  implications  of  risk  and  potential  capability 
over  a  specified  time  horizon. 
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Figure  7.  Dynamic  Portfolio  Extension 

Figure  7  illustrates  the  generalized  extension  of  the  objective  function  in  the  robust  portfolio 
formulation  shown  in  Equation  1 9  to  include  the  potential  effect  of  current  decisions  on  expectations 
of  future  portfolio  performance.  Decisions  can  include  the  myriad  contracting  and  pricing  initiatives 
used  to  control  costs,  improve  warfighter  portfolio  performance,  and  mitigate  unnecessary  risk 
throughout  the  acquisition  management  phase.  A  good  policy  will  also  exploit  potential  adaptive 
traits  in  collections  of  military  assets  where  the  associated  costs  of  evolving  requirements  are 
minimized  due  to  the  adaptability  of  the  chosen  set  of  assets.  The  complexity  arises  in  the  interlinked 
nature  of  sequential  decisions  where  the  consequences  of  earlier  decisions  can  affect  the  efficiency, 
cost  effectiveness,  and  ability  of  military  systems  to  adapt  to  new  requirements. 
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